When a host system (i.e., a user) is writing data to a persistent memory device or storage system, such as a magnetic disk system, the storage system may employ its own file system that controls transfer of IO (i.e., input and output) data between the host system and the storage system. The file system keeps track of various pieces of information about the IO data, which may be known as metadata. Metadata is information about the data, such as inode tables, block metadata (BMD), directories and mappings for finding various parts of the data, slice mapping, special block data, file type, and other forms of information about the data.
When a storage processor (SP) file system transfers data it also locally stores metadata in a cache in the SP. During IO processing, the SP file system writes to a log buffer memory location and marks the buffer as dirty, meaning that the data in the buffer is not the same as the data in the storage system, typically a persistent memory or non-volatile data memory media such as a magnetic hard disk drive (HDD), an electronic flash drive (EFD), or a solid state disk (SSD). That is to say that the data is not synchronized between the persistent memory of the storage system and the log buffer of the SP file system cache memory. To make the metadata consistent the SP file system may perform what may be known as a sync operation at time intervals, for example every 10 to 40 seconds. The sync operation is a period of time when the metadata in the SP file system cache is transferred to the storage media to update the metadata in the storage media, thus making the metadata consistent between the storage media and the SP cache, and synchronizing the data. The sync process may be called flushing the dirty metadata.
Sync operations require that a selected portion of the total IO process bandwidth be dedicated to the IO needed for the sync operation, and thus may reduce the total number of TO processes available for host R/W (read and write) requests. This may increase the host memory latency time for memory operations in high usage situations.